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DETAILED ACTION 

1. This Office action for U.S. Patent Application 10/802,428 is responsive to the 
Appeal Brief filed 20 November 2008. Currently, claims 1-3, 5-15, 17-25, 27, and 29 
are pending. 

2. In the previous Office action, claims 1-3 and 5-12 were rejected under 35 U.S.C. 
101 as non-statutory. Claims 1-3, 5-7, 10-15, 17-19, and 22-25 were rejected under 
35 U.S.C. 103(a) as obvious over "Motion-based Segmentation Using a Thresholded 
Merging Strategy on Watershed Segments" (de Smet et al.) in view of "K-Harmonic 
Means-A Data Clustering Algorithm" (Zhang et al.). Claims 8, 9, 20, and 21 were 
rejected under 35 U.S.C. 103(a) as obvious over de Smet et al. in view of Zhang et al. 
and in view of "A Video Segmentation Algorithm for Hierarchical Object Representations 
and its Implementation" (Herrmann et al.). Claims 27 and 29 were rejected under 35 
U.S.C. 103(a) as obvious over de Smet et al. in view of Zhang et al., and in view of US 
Patent 6,084,91 2 A (Reitmeier et al.). 

3. In view of the Appeal Brief filed on 20 November 2008, PROSECUTION IS 
HEREBY REOPENED. New grounds of rejection under 35 U.S.C. section 101 are set 
forth below. 

To avoid abandonment of the application, appellant must exercise one of the 
following two options: 
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(1) file a reply under 37 CFR 1.111 (if this Office action is non-final) or a reply 
under 37 CFR 1 .1 13 (if this Office action is final); or, 

(2) initiate a new appeal by filing a notice of appeal under 37 CFR 41 .31 followed 
by an appeal brief under 37 CFR 41 .37. The previously paid notice of appeal fee and 
appeal brief fee can be applied to the new appeal. If, however, the appeal fees set forth 
in 37 CFR 41.20 have been increased since they were previously paid, then appellant 
must pay the difference between the increased fees and the amount previously paid. 

A Supervisory Patent Examiner (SPE) has approved of reopening prosecution by 
signing below: 

/Mehrdad Dastouri/ 

Supervisory Patent Examiner, Art Unit 2621. 

Response to Arguments 

4. Applicant's arguments filed with respect to the rejection of claims 1-3 and 5-12 
under 35 U.S.C. 101 as non-statutory have been fully considered but they are not 
persuasive. The examiner respectfully disagrees with Applicant's assessment that the 
claimed "program storage device" does not encompass the non-statutory embodiments 
described in the specification. The specification in paragraph 0088 states that the 
process is "embodied in a process-readable medium or carrier, e.g. one or more of the 
fixed and/or removable data storage devices 588 illustrated in Fig. 5, or other data 
storage or data communications devices". This language implies that both the 
"processor-readable medium" and the "carrier" embody the "fixed and/or removable 
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data storage devices", the "other data storage", and the "data communications devices". 
Although the data storage device 588 is shown in figure 5 to be embodied as a statutory 
floppy disk, the term "other data storage" may encompass both statutory and non- 
statutory embodiments. 

Applicant additionally states that applying Nuijten to claim 1 was improper, since 
Nuijten held a "storage medium having stored thereon a signal with embedded 
supplemental data" to be statutory (pg. 1498). However, in the specification of 
application 09/211,928, the only storage medium described therein was an optical disc 
according to the DVD-Audio specification (pg. 2: line 18; pg. 6: line 13). Then, the 
"storage medium" in claim 15 was only described in a statutory embodiment. This 
differs from the present invention, in which the claimed "program storage device" itself is 
not limited to the "data storage device 588". Accordingly, the narrow interpretation of 
Nuijten as only relevant to a claimed signal perse and not the claimed "program storage 
device" is improper due to the broader recitation of storage media in the present 
application than in the Nuijten application. 

5. Applicant's arguments filed with respect to the rejection of claim 1 under 35 
U.S.C. 103(a) as obvious over de Smet et al. and Zhang et al. have been fully 
considered but they are not persuasive. Applicant makes two arguments: first, that the 
present invention performs multiple K regression functions, whereas the Zhang 
reference discloses a single performance function, and second, that it was improper to 
combine the De Smet and Zhang references. 
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Regarding the allegation that the present invention differs from the Zhang 
reference, it is respectfully submitted that in Zhang, the recursive formula for m(k) is 
performed for K values of m (page 5), where K is the number of centers. This formula, 
in turn, includes factors of distance d and location of data point x. Generally speaking, a 
regression function is a function Y=f(X,Z), where Y is a dependent variable, X is an 
independent variable, and Z is an unknown error or residual variable. In Zhang, then, 
the calculation of m is a regression function, based on independent variable x and 
residual variable d. Since the function for determining m is performed K times, it is 
respectfully submitted that Zhang discloses multiple regression functions, one for each 
of K centers m. 

Regarding the allegation that the combination of the two references was 
improper, the appeal brief states, "There is no apparent evidence that De Smet teaches 
a center-based approach for obtaining a motion-based segmentation of an image 
sequence" (pg. 11), and "De Smet's segmentation method does not use centers" (pg. 
12). However, the appeal brief also states in page 9 that "De Smet teaches using a 
direct K-means (KM) cluster algorithm" which is then described as "a center-based 
iterative algorithm that refines the clusters defined by K centers". Therefore, the 
argument that it would be improper to apply the "center-based clustering" of ZHM (pg. 
12) to de Smet is based on a contradiction, and thus invalid. 



Claim Rejections - 35 USC § 101 
6. 35 U.S.C. 101 reads as follows: 
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Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

7. Claims 1-3, 5-12, and 25 are rejected under 35 U.S.C. 101 because the claimed 
invention is directed to non-statutory subject matter. 

Claims 1-3 and 5-12 are drawn to a "program storage device" storing functional 
descriptive material. Normally, the claims would be statutory. In re Nuijten, 84 USPQ2d 
1495, 1498) (Fed. Cir. 2007). However, the specification, at page 0088, defines the 
claimed storage as encompassing statutory material such as a floppy disk, as well as 
non-statutory subject matter as a "carrier" or "communications devices". A signal 
embodying functional descriptive material is neither a process nor a product (i.e., a 
tangible "thing"), and therefore does not fall within one of the statutory classes of §101 . 
Rather, a "signal" is a form of energy, in the absence of any physical structure or 
tangible material. Id. at 1500. 

Regarding claim 25, Supreme Court precedent 1 and recent Federal Circuit 
decisions 2 indicate that a statutory "process" under 35 U.S.C. 101 must (1) be tied to 
another statutory category (such as a particular apparatus), or (2) transform underlying 
subject matter (such as an article or material) to a different state or thing. While the 
instant claim(s) recite a series of steps or acts to be performed, the claims neither 
transform underlying subject matter nor positively tie to another statutory category that 
accomplishes the claimed method steps, and therefore do not qualify as a statutory 



1 Diamond v. Diehr, 450 U.S. 175, 184 (1981); Parker v. Flook, 437 U.S. 584, 588 n.9 (1978); Gottschalk v. Benson, 
409 U.S. 63, 70 (1972); Cochrane v. Deener, 94 US 780, 787-88 (1876). 

2 In reBilski, 88 USPQ2d 1385 (Fed. Cir. 2008). 
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process. In the present invention, method claim 25 does not state what performs the 
claimed method steps of providing data points and performing regression clustering. 

Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

9. Claims 1-3, 5-7, 10-15, 17-19, and 22-25 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over "Motion-based Segmentation Using a Thresholded Merging 
Strategy on Watershed Segments" (de Smet et al.) in view of "K-Harmonic Means-A 
Data Clustering Algorithm" (Zhang et al.). De Smet et al. discloses using an iterative 
segment-merging technique to determine information for an image (abstract). 

Regarding claims 1,13, and 25, in de Smet et al., an initial motion field is first 
determined with a block-matching technique on 4 x 4 blocks (§ 2.1). These initial block 
motion vectors are used for the initial segmentation (§ 2.3). Then, the step of 
performing the block-based motion estimation is the claimed step of "providing data 
points". Next, the segments are iteratively merged according to similar or shared 
motion, according to the K-means clustering algorithm (§ 2.3). This is the claimed step 
of "clustering the data points". When this process is finished, the result is a series of 
large segments corresponding to distinct moving regions of an image, each with an 
associated motion vector (§ 2.3). This is the claimed step of "providing motion 
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estimation". However, the present invention specifies performing regression clustering 
according to a K-Harmonic Means function, which is not the same as the K-means 
function of de Smet et al. 

Zhang et al. discloses the K-Harmonic Means data clustering algorithm. 
Regarding claim 1, 3, and 25, Zhang et al. teaches selecting K centers m(l) from N data 
points x(i) (pg. 1), initializing center points (pg. 2) and performing an initial iteration (pg. 
5), calculating distance d(i,l) between data point x(i) and center point m(l) (pg. 4), 
calculating membership probability q(i,k) and weighting function p(i,k) (pg. 5), 
recursively calculating new m(k), (pg. 5), and stopping when the recursively-calculated 
performance value stabilizes, that is, when its change with each iteration becomes small 
(pg. 5). Then, the K calculations of centers m(l) are the claimed "regression functions" 
for performing regression clustering according to the K-harmonic means function. 

De Smet et al. discloses the claimed invention, except for using K-Harmonic 
Means function to perform regression clustering. Zhang et al. teaches that it was known 
to perform data clustering with the K-Harmonic means function. Therefore, it would 
have been obvious to one having ordinary skill in the art at the time the invention was 
made to perform clustering based on a K-Harmonic means function, as taught by Zhang 
et al., rather than a linear function such as the K-means function of de Smet et al., since 
Zhang et al. states in the abstract that K-harmonic clustering is less sensitive to 
detrimental effects from sub-optimal initialization than conventional clustering 
techniques. Additionally, the K-harmonic means function was specifically designed as 
an improved version of the K-means function, which "significantly improves the quality 
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of clustering results" compared with K-means (abstract). Then, the de Smet et al. 
algorithm contains a basic method which the present invention is an improvement 
thereof. The prior art Zhang et al. describes a known technique, the K-Harmonic means 
clustering algorithm, applicable to the base segmentation method of de Smet et al., by 
substituting it for the K-means algorithm. Then, one having ordinary skill in the art 
would have recognized that applying the K-harmonic means algorithm of Zhang et al. to 
de Smet et al. would have yielded the predictable result of "significantly [improved] 
quality of clustering results" and resulted in an improved system. Therefore, it is 
respectfully submitted that the use of K-harmonic clustering in de Smet et al. is 
considered obvious, since it has been held that applying a known technique to a known 
method ready for improvement to yield predictable results involves only routine skill in 
the art. Dann v. Johnston, 425 U.S. 219, 230, 189 USPQ 257, 261 (1976); In re Nillsen, 
851 F.2d 1401, 1403, 7 USPQ2d 1500, 1502 (Fed. Cir. 1988). 

Regarding claims 2, 3, 14, and 15, as previously mentioned, de Smet et al. 
produces a motion vector for each segment in an image (§ 2.3). As a result, the most 
important moving areas are determined (§ 3). Regarding claims 10 and 22, in de Smet 
et al., pixels are set as (x,y,t) triples, with x and y as spatial coordinates and t as a time 
coordinate (§ 2.2). 

Regarding claims 11 and 23, de Smet et al. illustrates motion fields (figures 3-6). 
Although these motion fields are not shown as overlaid on the images, the examiner 
takes Official Notice that it was well-known in the art at the time of the invention to 
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display a motion field superimposed on an image to provide a visual representation of 
motion vectors. 

Regarding claims 12 and 24, de Smet et al. illustrates highlighted motion 
segments overlaid on an image (figures 11 and 12). 

Regarding claims 5 and 17, in Zhang et al., a clustering in which initialization is 
randomized is described (pg. 11). 

Regarding claims 6 and 18, in Zhang et al., KHM is described as a recursive 
function that performs until stabilization is achieved (pg. 5). 

Regarding claims 7 and 1 9, insensitivity to initialization is an inherent result of the 
K-Harmonic means algorithm (abstract). 

10. Claims 8, 9, 20, and 21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over de Smet et al. in view of Zhang et al. as applied to claims 1 and 13 
above, and further in view of "A Video Segmentation Algorithm for Hierarchical Object 
Representations and its Implementation" (Herrmann et al.). Claims 8, 9, 20, and 21 
disclose using color information to segment images, but de Smet et al. only discloses 
"standard watershed techniques" (§ 2.2) to perform the initial segmentation without 
providing details. 

Herrmann et al. discloses a method for image segmentation to extract objects 
from a moving image. Regarding claims 8, 9, 20, and 21, after an initial block-matching 
motion estimation, similar to de Smet et al., images are segmented according to specific 
color information, followed by shape analysis, and lastly motion analysis to merge 
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regions to determine objects (§ II). This color, shape, and motion information form the 
claimed "predetermined criteria". In color analysis, a region is determined as 
homogeneous if the pixel difference in the region is below a threshold. Homogenous, 
connected areas are determined as "quasi-flat zones". These quasi-flat zones are 
further processed and become the basis for further segmentation (§ II. B). Then, the 
color analysis is the claimed step of "portioning data according to color". 

De Smet et al., in combination with Zhang et al., disclose a majority of the 
features of claims 8, 9, 20, and 21 as discussed above, the claimed invention except for 
color segmentation. Herrmann et al. teaches that it was known to segment a moving 
image according to color. Therefore, it would have been obvious to one having ordinary 
skill in the art at the time the invention was made to determine image segmentation by 
color as taught by Herrmann et al., since Herrmann et al. teaches in page 205, third 
paragraph, that color analysis produces the most accurate type of segmentation. 

11. Claims 27 and 29 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over de Smet et al. in view of Zhang et al., and in view of US Patent 6,084,912 A 
(Reitmeier et al.) Claims 27 and 29 are in means-plus-function format, complying with 
35 U.S.C. 112, sixth paragraph. Accordingly, the "system" of claims 27 and 29 will be 
limited to a general-purpose computer such as a PC, as illustrated in figure 5 of the 
specification of the present invention. Although it is implied that the algorithms of de 
Smet et al. and Zhang et al. are computer-operated, neither de Smet et al. nor Zhang et 
al. explicitly teach this. 
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Reitmeier et al. discloses a video encoder. This encoder may operate on 
MPEG-4 video (column 1: line 57), as specified in paragraphs [0006] and [0007] of the 
present invention as a codec on which the present invention is applied. Regarding 
claim 27, the encoder of Reitmeier et al. may operate as a software application on a 
general-purpose computer (column 2: lines 64-67). Regarding claim 29, Reitmeier et al. 
discloses frame memory 155, which stores decoded reference frames for motion 
compensation (column 7: line 27), as is well-known in MPEG encoders. 

De Smet et al., combined with Zhang et al., discloses the claimed invention 
except for encoding video on a general-purpose computer. Reitmeier et al. teaches that 
it was known to implement an MPEG-4 encoder as software embedded on a computer. 
Therefore, it would have been obvious to one having ordinary skill in the art at the time 
the invention was made to encode the system of de Smet et al. and Zhang et al. in a 
software MPEG encoder embedded on a computer, as taught by Reitmeier et al., in 
order to perform computationally complex functions such as motion compensation, 
quantization, and variable-length encoding inherent in the video coding process. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David N. Werner whose telephone number is (571)272- 
9662. The examiner can normally be reached on Monday-Friday from 10:00-6:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on (571) 272-7418. The fax phone 
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number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



ID. N. W./ 

Examiner, Art Unit 2621 
/Mehrdad Dastouri/ 

Supervisory Patent Examiner, Art Unit 2621 



